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Abstract 

A central problem in signal processing and communications is to design signals that are compact both in 
time and frequency. The variance is accepted as a good measure of compactness, and with this defini- 
tion Heisenberg states that a given function cannot be arbitrarily compact both in time and frequency, 
defining an "uncertainty" lower bound. In continuous-time, Gaussian functions reach this bound. For se- 
quences, however, this is not true; it is known that Heisenberg's bound is generally unachievable. For a 
chosen frequency variance, we formulate the search for maximally compact sequences as an exactly and 
efficiently solved convex optimization problem, thus providing a sharp uncertainty principle for sequences. 
Interestingly, the optimization formulation also reveals that maximally compact sequences are derived from 
Mathieu's harmonic cosine function of order zero. We further provide rational asymptotic expansions of this 
sharp uncertainty bound. 

Keywords: Compact sequences, Heisenberg uncertainty principle, Time Spread, Filter design, Mathieu 
function. Discrete sequences. Circular statistics. Semi-definite relaxation 



1. Introduction 

Suppose you are asked to design filters that are sharp in the frequency domain and at the same time 
compact in the time domain. The same problem is posed in designing sharp probing basis functions with 
compact frequency characteristics. In order to formulate these problems mathematically, we need to have 
a correct and universal definition of compactness and clarify what we mean by saying a signal is spread in 
time or frequency. 

These notions are well defined and established for continuous-time signals [1, 2] and their properties are 
studied thoroughly in the literature. For a continuous-time signal, we can define the time and frequency 
characteristics of a signal as in Table 1. Note the connection of these definitions with the mean and variance 
of a probability distribution function |a::(i)p/ The value of Aj is considered as the spread of the signal 

in the time domain while represents its spread in the frequency domain. We say that a signal is compact 
in time (or frequency) if it has a small time (or frequency) spread. 

The Heisenberg uncertainty principle [1] states that continuous-time signals cannot be arbitrarily compact 
in both domains. Specifically, for any x{t) € i^(K), 

rye = A?A2^>i, (1) 

where the lower bound is achieved for Gaussian signals of the form x{t) — 76^"*^, a > [2]. The subscript 
c stands for continuous-time definitions. We call rjc the time-frequency spread of x. 
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Table 2: Time and frequency centers and spreads for a discrete time signal Xn as extensions of Table 1 [2]. 



Although the continuous Heisenberg uncertainty principle is widely used in theory, in practice we often 

work with discrctc-timc signals (e.g. filters and wavelets). Thus, equivalent definitions for discrete-time 
sequences are needed in signal processing. In the next section we study two common definitions of center 
and spread available in the literature. 

1.1. Uncertainty principles for sequences 

An obvious and intuitive extension of the definitions in Table 1 for discrete-time signals is presented in 
Table 2, where 

X(e^") = x„e-J"" w e M , (2) 
nez 

is the discrete-time Fourier transform (DTFT) of a;„. 

Using the definitions in Table 2 [2], we can also state the Heisenberg uncertainty principle for discrete- 
time signals as 

m = AlAl^>^, XnGe\Z)withXien = 0, (3) 

where the subscript £ stands for linear in reference to the definition of the frequency spread. Note the 
extra assumption on the Fourier transform of the signal in (3). This assumption is necessary for the result 
to hold. 

Example 1. Take Xn = Sn + 75„_i + 2Sn-2- It is easy to verify that that |X(e-''^)| = 0.22 ^ 0, which 
violates the condition X{e^'^) = 0. The linear time-frequency spread of this signal according to Table 2 is 
m = 0.159 < 1/4. 

In addition to the restriction on the Heisenberg uncertainty principle, the definitions in Table 2 do 
not capture the periodic nature of X(e-'") for the frequency center and spread. In the search for more 
natural properties, we can adopt definitions for circular moments widely used in quantum mechanics [3] and 
directional statistics [4]. 

For a sequence a;„, n £ Z, with a 27r periodic DTFT, X{e^'^) as in (2), the first trigonometric moment 
is defined as [5, 6] 

^(^) = -TTl^ 

ZTT \\x\\ 

The first trigonometric moment was originally defined for probability distributions on a circle. With 
proper normalization, this definition applies also to periodic functions. 



ei'^\X{e^")fdoj. 



(4) 
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Tabic 3: Time and frequency centers and spreads for a discrete time signal Xn using circular moments, where t{x) is defined 
in (4). 
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Figure 1; Time- frequency spread of continuous vs. sampled Gaussians. The solid line shows the 1/4 Heisenberg bound 
which is achieved by continuous Gaussian signals (with definitions in Table 1), while the markers show the time- frequency spread 
(according to Table 3) of sampled Gaussian sequences. The question is if the gap between the two curves is inherited from the 
properties of sequences or sampled Gaussians are not optimal in discrete domain. 



Using (4), the periodic frequency spread is defined as [3]: 

l-|r(x)|2 



A' = 



- 1. 



(5) 



\rixW 

The definition of remains unchanged as in Table 2. These definitions are summarized in Table 3. 
1.2. Contribution 

In this paper, using the definitions in Table 3, we revisit the Heisenberg uncertainty principle for discrete- 
time signals. 

Wc address the fundamental yet unanswered question: If someone; asks us to design a discrete filter with 
a certain frequency spread (A^^ fixed), can we return the sequence with minimal time spread A^? In other 
words, the problem is to find the solution to 



A^,opt = niinimize A^ 



subject to A^ = fixed . 



(6) 
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In order to provide an insight on the uncertainty principle in the discrete-time domain, we do a simple test. 

In figure 1 we show the time-frequency spread of continuous Gaussian signals by the solid line on 1/4. This 
is the Heisenberg's uncertainty bound which is achievable by continuous-time Gaussians. Further, using the 
definitions in Table 3, we compute the time-frequency spread of sampled Gaussian sequences. As stated 
before, the time-frequency spread tends to 1/4 as the frequency spread of Gaussians decreases, but when 
the frequency spread is large, the values are far from the uncertainty bound. Two questions arise here; is 
the 1/4 uncertainty bound also tight for discrete sequences? and are sampled Gaussians the minimizers of 
the uncertainty in the discrete domain. Answers to these questions will be apparent if we can solve (6). We 
call the solution of (6) a maximally compact sequence. 

Framing the design of maximally compact sequences as an optimization problem, we show that contrary 
to the continuous case, it is not possible to reach a constant time-frequency lower bound for arbitrary time 
or frequency spreads. We further develop a simple optimization framework to find maximally compact 
sequences in the time domain for a given frequency spread. In other words, we provide in a constructive 
and numerical way, a sharp uncertainty principle for sequences, later shown in Figure 4. Wc also show that 
the Fourier spectra of maximally compact sequences are in fact a very special class of Mathieu functions. 
Using the asymptotical expansion of these functions, we develop closed-form bounds on the time-frequency 
spread of maximally compact sequences. 

1.3. Related Work 

The classic uncertainty principle [1] assumes continuous-time non-periodic signals. Several works in 
the signal processing community also address the discrete-time/discrete- frequency case [7, 8, 9]. Our work 
bridges these two cases by considering the discrete-time/continuous- frequency regime. 

Note that not all studies about the uncertainty principle concern the notion of spread. For example, the 
authors in [8] propose the uncertainty bound on the information content of signals (entropy) and [9] provides 
a bound on the non-zero coefficients of discrete-time sequences and their discrete Fourier transforms. 

The discrctc-timc/contimious- frequency scenario has been recently encountered in many practical ap- 
plications in signal processing. Examples include uncertainty principle on graphs [10] and on spheres [11]. 
Sudies on the periodic frequency spread can be found in [3] and [12]. The most comprehensive work on the 
uncertainty relations for discrete sequences is found in [6] . The authors show that 1/4 is a lower-bound on 
the time-frequency spread, which can only be achieved asymptotically as the sequence spreads in time. We 
provide sharp achievable bounds in the non-extreme case which match the results in [6] in asymptotic. 

This problem is similar to although different than — the design of Slcpian's Discrete Prolate Spheroidal 
Sequences (DPSS's). First introduced by Slepian in 1978 [13], DPSS's are sequences designed to be both 
limited in the time and band-limited in the frequency domains. For a finite length, N in time and a cut-off 
frequency W, the DPSS's arc a collection of discrete-time sequences that arc strictly band-limited to the 
digital frequency range |/| < W, yet highly concentrated in time to the index range n = 0,1, - ■ ■ ,N—1. Such 
sequences can be found using an algorithm similar to the Papoulis-Gerchberg method. Note the difference 
of such sequences to the ones that wc intend to design in this work; wc do not impose any constraints on 
the bandwidth of the sequences in the frequency domain. Also, the original ideas presented in this work 
are applicable both to finite and infinite length sequences. Moreover, we focus on the concentration of the 
signals in the time and frequency domain using the notion of variance. 

1.4. Organization 

The organization of the paper is as follows. In Section 2, we see how the definitions in Tables 2 and 3 
are found starting from the continuous-time definition and using the classic notion of time and frequency 
uncertainties. In Section 3, we formulate precisely the problem of finding maximally compact sequences. In 
Section 4, we first establish key properties of maximally compact sequences to narrow down the search to 
real positive sequences, which then allows to setup and solve the problem nrumerically. From the numerical 
formulation, we identify the spectrum of maximally compact sequences. Finally after the simulation results 
and conclusions, appendices contain proofs of the stated results. 
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2. Theory 



The choice of A^^ as an angular variance must be motivated. A thorough study on the subject in [3] 
provides many heuristic reasons. If these reasons all have their merits, we would also like to show that 
the definition of a frequency spread is unambiguously linked to a pseudo-differential operator, providing a 
simple and unique selection criterion. We will show that A^^ corresponds to the finite difference operator, 
which is the simplest among discrete diffrentials. 

The Heisenberg uncertainty principle is rooted in particle physics, and therefore thought in terms of 
position-momentum uncertainty rather than time-frequency spread as in the signal processing community. 
These two interpretations are equivalent for continuous signals, but the intuitions behind them are useful 
to make the transition from continuous to discrete time. 



2.1. Uncertainty Principle for Linear Operators 

Consider a Hilbcrt space TL with the inner-product (• , •) and the induced norm |ja;|l '= ^ {x , x) . Define 
self-adjoint linear operators L, M : H i-^ H, and their "centered" counterparts. Note that the centered 
operators are not linear, nor self-adjoint. 

Lx {L — iil{x))x, where ^Jil{x) =' (La; , a;) / ||a;||^ . 

Assuming L is self-adjoint, /i^ (.x) G R. 

For any pair of linear operators, the commutator 

[L , M] LM-ML, 

vanishes if and only if its arguments commute [14]. 

With these definitions, the Heisenberg uncertainty principle is 

Theorem 1. (Schrodinger 1930 [15]) 

\\Lxf \\Mxf > ^\{[L, M]x, x)f . (7) 
Proof. Using Cauchy-Schwarz inequality and the self-adjointness of L and M, 



\Lx\\^ \\Mxf > \{Mx , Lx)\ 



(LMx , x)~ HL {x) jjLM {x) \\x\Y 



11 9 

- {{LM - ML)x , a;) + - {{LM + ML)x , x) - hl (x) m {x) \\x\\'' 



The two halves within the modulus are respectively imaginary and real, therefore with the anticommutator 



{L , M} = LM + ML, one obtains the Schrodinger uncertainty principle 



\\Lx\\ \\Mx\\ > -^\{[L, M]x, x)\ {{L, M}x, x) -2ij,l{x)ij-m{x)\\x\\ 

from which the weaker Heisenberg uncertainty principle immediatly follows. 



□ 



2.2. The journey from, continuous to discrete 

We apply the general uncertainty relation (7) to continuous and discrete-time signals to make some 
connections explicit. In particular, we study two schemes to make the transition from continuous-time to 
discrete-time. 
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2.2.1. Continuous time-frequency uncertainty 

Let x{t) e i2(K), the locaUzation L and momentum M operators are 

Lx{t) =U-x{t) , 
«x(*) «§(*). 

Moreover, because of the similarity between the continuous-time domain and the CTFT domain, localization 
and momentum are Fourier dual of each other : 

Lx{t) i — > J-i— (w) = MjrX{uj) , 
auj 

Mx{t) jwX(w) =^ LrX{uj) . 

This duality shows that the localization-momentum and time-frequency localization interpretations are 
the two sides of a same coin. 
With the evaluation of 

\{[t,d/dt]x,x}\^ = \\xf , 

and use of Parseval's equality 

\\L-^X\\'/\\Xf = \\Mx\\'/\\xf , 
the uncertainty relation (7) yields the continuous-time Heisenberg uncertainty principle (1) 

rjc = A? A^^ > 1 , 

1 1 2 / 2 1 1 2 / 2 

where A| = ||ia;|| / ||a;|| and A^^ = ||L^X|| / ||X|| as found in Table 1. 

2.2.2. First attempt at discrete-time uncertainty: the DSP point of view 

In the same way that we defined the localization and momentum in time and frequency for continuous- 
time signals, wc define them for sequences. The process can be seen either from the time-domain as a 
discretization or from the frequency domain as a periodization. 

If we follow the time-frequency interpretation, it is natural to discretize L and periodize Ljr 

Designed Localization: Implied Momentum: 

Lxn = n-xn DTFT J^i^'") = M^X{e^'-) (8) 

L^X(e^-) "^^^ w-X{e^'-) ^ ^ * = Mxn 

where u represents the sawtooth wave with period 27r. The implied frequency-domain momentum Mjf 
has the properties one would expect for momentum as it measures the variation of the spectrum locally 
by differentiation; on the other side, the implied time-domain momentum M does not lead to an easy 
interpretation. 

This choice of operators not only causes interpretation problems. The uncertainty relation is 



2 a2 -1 
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m = A^ A^ > 



1 - 



|X(e^'^)|^ - 2Re{X'(e^'^)X*(e^'^)} 



\xp 



(9) 



where A^ ll^ \\Lxf / \\xf and A^^ = \\Lj.xf / \\xf, as found in Table 2. 
Under the condition X{e^'^) = 0, the uncertainty principle (3) is met 

m = AlAl>\, forX(e^-)=0. (10) 

However the necessity of a root at tt severely reduces the utility of this result, exluding sequences such as 
the discretized gaussian kernel. 




Figure 2: Two ways to obtain an uncertainty principle for sequences. On the left side a) — by periodization — one 
obtains the definitions of Table 2, on the right side b) — by discretization — one obtains the definitions of Table 3. Note that 
periodization and discretization yield the same result for the top plane, but not for the bottom plane. 

2.2.3. Second attempt at discrete-tim,e uncertainty: the physicist point of view 

Failure of the first attempt can be pinned down on the definition of the frequency-domain localization. 
Instead of designing both localization operators, we may design both time-domain operators. The task is 
thus to discretize localization (already done) and momentum. For the latter, the finite difference filter is a 
natural candidate 

Designed in the time-domain Implied in the frequency-domain 

Lxn = n-xn DTFT Jl^i^'^) = (11) 

MXn = Xn-Xn-l ^ (1 - 6"^'^) • X(e^'^) = L^X{e^^) 

The implied operator Lj: has intuitive properties. For example, 

lli^xf =2 j (1 - cos(a;)) |X(e^")|^rfw , 

shows that the spread is measured with respect to the smooth kernel 2(1 — cos(a;)) instead of oP in (8). 

Note that 2(1 — cos(a;)) = + 0{ijj'^), so the two definitions of spread coincide for w — > 0, and should 
yield asymptotically equal results for sequences with a narrow spectrum. 

The operators Mjr and Ljr arc the ones most often used in directional statistics [4]. 

The right-hand side of (7) evaluates to 

\{[l-e-^- , jd/dLo\X ,X)\^ = \\xf\T{x)\' . (12) 
For \t{x)\ ^ — which is equivalent to ||x||q > 1 — the corresponding uncertainty relation [3] is therefore 

rip = ^l^l,>\, for|N|o>l, (13) 



where A,^, = ..iLfja,, y||2 and are as found in Table 3. 



def ll-L^^I 

\r(X)\^\\Xr 

The study of continuous and discrete uncertainty principle shows that the definitions of time and fre- 
quency spreads found in Table 3 are not arbitrary and follow from the most natural discretization of the 
continuous-time definitions found in Table 1. 
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3. Problem Statement 

As mentioned in Section 1.2, the main objective of this paper is to design maximally compact sequences 
as solutions of (6). Thus we are interested in solving 

^n,opt = minimize 

(14) 

subject to A^^ = . 

where is the fixed, given frequency spread of the sequence. We saw in (13) that the time-frequency 
spread of such sequences is naturally bounded from below by the Heisenberg uncertainty bound. Prestin et 
al. in [5] show that the lower bound in (13) is achievable only asymptotically when the frequency spread of 
the sequence tends to zero. Thus, the question is "what is the minimal achievable uncertainty for sequences 
with a given spread?". The answer to this question lies in the solution of (14). 

Although finding a tight bound for maximally compact seqiiences is an important task, the more in- 
teresting problem is to constructively design sequences which achieve such bound. The ideal answer to 
this question would be to find analytic closed form solutions — like Gaussian signals for the continuous time 
problem — achieving the tight bound. In the following we will address these problems. 



4. Main Results 

We start with some properties of maximally compact sequences. These properties will greatly facilitate 

the task of solving (14). 

Jf..l. Properties of Maximally Compact Sequences 

In the definitions of time and frequency spreads in Table 3 we considered complex sequences and their 
DTFTs. In the following, we establish two lemmas that make the search for maximally compact sequences 
easier. 

Lemma 1. Maximally compact sequences are generalized linear-phase sequences derived from real- valued, 
positive maximally compact sequences; i.e. a;„ is a maximally compact sequence only if 

3'fi,tp€ [0, 27r[ such that a;„ = |x„|e-^(^"+*) , (15) 

where \x\ is a maximally compact sequence. 

Proof. See Appendix A.l. □ 
Consider also the shift operator 

Xn+. e^'^^X(e^'^) , i^eR, (16) 

whose principal effect is to shift the time centre of a sequence 

fln{Xn+u) = tJ.n{x) - V. (17) 

Notice that u might be non-integer, in which a shorthand for sine resampling on a grid 

shifted by v in the time-domain. 

Lemma 2. If a; is a maximally compact sequence, then Xn-n„(x) is also maximally compact. 

Proof. See Appendix A. 2. □ 

Lemmas 1 and 2 greatly reduce the complexity of the problem, and from here on we only consider — 
without loss of generality — real, positive sequences x, with /i„(a;) = and ||a;||^ = 1. 
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4-2. Main Theorems 

The following theorem is the core of the results presented in this paper. 

Theorem 2. For finding the solution of (14), it is sufficient to solve the following semi-definite program 
(SDP) 



minimize tiiAX) 

X 

subject to tr(JBX) = a 

tr(X) = l, X^O, 



(18) 



Further, X°p*, the solution to (18) has rank one and X°p* = a;°P* a;°P' , with a;°P* the 



where a = , , 
solution of (14). Matrices A and B are defined as 



A = 



B = 



(19) 



Proof. See Appendix B. 



□ 



The SDP in (18) can be solved to an arbitrary precision by using existing approaches in the optimization 
literature; for example using the cvx software package [16]. This, gives a constructive way to design sequences 
that are maximally compact in the time domain with a given frequency spread. 

Example 2. Take cr^ = 0.1 to be the fixed and given frequency spread of the sequence. We can use 
cvx to solve the semi-definite program (18) and find the optimal value of = 2.62. This results in the 
time-frequency spread of r]p = 0.262. The simple code in MATLAB is: 

cvx_begin 
variable X(n,n) ; 
minimize (trace (A*X) ) 
subject to 

trace (B*X) == l/sqrt(l+0. 1) 

trace (X) == 1 

X == semi-definite (n) 
cvx_end; 

Note that contrary to continuous-time signals, we cannot reach the 0.25 lower bound for sequences. The 
resulting sequence and its DTFT are shown in Figure 3. 



The dual of SDP (18) is [17]: 



maximize a Ai + A2 

Ai ,A2 



subject to A - Ai B - A2 / ^ 
Lemma 3. For the primal problem (18) and the dual (20), strong duality holds. 
Proof. We refer the reader to Appendix C for the proof. 



(20) 



□ 
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Figure 3: An example solution of (18). The output of the SDP in (18) with cr^ = 0.1 using cvx in Example 2. The optimal 
value for is found to be 2.62 which results in a time-frequency spread of rjp = 0.262. 



Thus, for finding the time-frequency spread of maximally compact sequences, solving the dual problem 
suffices. 

Note that although Theorem 2 provides a constructive way for finding maximally compact sequences, 
it does not specify the closed form for these sequences. One would be interested to see if — in analogy to 
continuous-time — sampled gaussians are maximally compact? The answer is negative, as shown by this 
theorem: 

Theorem 3. The DTFT spectrum, X(e^") of maximally compact sequences are Mathieu functions. More 
specifically, 

X{en = 70 • ceo(-2Ai ; (c. - iOo)/2)e^^'^ , (21) 

where |7o| = ||ceo(— 2Ai ; {lu — ujq)/2)\\^^, wq and fi are shifts in frequency or time and Ai is the optimal 
solution of the dual problem (20). ceo{q ;uj) is Mathieu's harmonic cosine function of order zero. 

For the proof of the theorem and further insights about Mathieu functions, we refer the reader to 
Appendix D. 

Using the constructive method presented in Theorem 2, we can find the achievable (and tight) uncertainty 
principle bound for discrete sequences. This is shown and discussed more in Section 5 and Figure 4. However, 
a numerically computed boundary may not always be practical, and even though the numerical solution 
exactly solves the problem, its accuracy may be challenged. Therefore, we characterize the asymptotic 
behavior of the time-frequency bound: 

Theorem 4. If Xn is maximally compact for a given A^^ = a^, then 

- > (l - . (22) 

Proof. The proof for this theorem is provided in Appendix E. □ 

This fundamental result states that for a given frequency spread, we cannot design sequences which 
achieve the classic Heisenberg uncertainty bound. We will see how this curve compares to the classic 
Heisenberg bound in Section 5. 

The lower bound in (22) converges to 1/2 as the value of cr^ grows, and "pushes up" the time-frequency 
spread of maximally compact sequences towards 1/2 which is also an asymptotic upper bound on the 
time- frequency spread as A^^ — > oo; indeed, one may construct the unit- norm sequence Xn'' — £^n+i + 
\/l — 2e'^5n + sSn-i, which verifies lime^o Vpi^^^^ ) — 1/2- 
Theorem 5. For small values of cr^, maximally compact sequences satisfy 
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Figure 4: New uncertainty bounds. The solid line shoves the results of solving the SDP in (18). The dotted line shows the 
classic Heisenberg uncertainty principle. The dashed lines show the analytic lower and upper bounds found in Theorems 4 and 
5, respectively. 



Proof. See Appendix F. □ 

For small values of a^, the upper bound in (23) converges from above to 1/4, thus "pushing down" the 
time-frequency spread of maximally compact sequences towards the Heisenberg uncertainty bound 1 /4 from 
above. 

4-2.1. Finite-Length Sequences 

The theory that we have provided so far holds for infinite sequences. For computational purposes, we 
have to assume finite length for the sequences in the time domain, which is not an issue if the sequence 
length is chosen to be long enough. As a side benefit, a length constraint on the sequence may be put at 
will without changing the design algorithm. 



5. Simulation Results 

In order to show the behaviour of the results obtained in Theorems 2, 4 and 5, we ran some simulations. 
For this end, we assumed that the designed filter is finite length with 201 taps in the time domain (the 
length is long enough not to pose restrictions on the solution) . For different values of A = , we solved 
the semi-definite program (18) using the cvx toolbox in MATLAB. 

The resulting values of A^^ were then multiplied with the corresponding A^^ to produce the time- 
frequency spread of maximally compact sequences. The time-frequency spread of maximally compact se- 
quences versus their frequency spread is shown with the solid curve in Figure 4. This means — numerically — 
that any time-frequency spread under this curve is not achievable. The dotted line in this figure shows 
the classic Heisenberg uncertainty bound. Comparing the two curves shows the gap between the classic 
Heisenberg principle and what is achievable in practice. The dashed lines represent analytical lower and 
upper bounds for the time- frequency spread of maximally compact sequences (found in Theorems 4 and 5, 
respectively) . 

Further, to give an insight on how the time-frequency spread of some common filters compare to that 
of maximally compact sequences, we plot their time-frequency spread together with the new uncertainty 
bound in Figure 5. By changing the length of each filter in time, we can find its time and frequency spreads 
which results in a point on the figure. We observe that as shown by Prestin et al. in [6], asymptotically 
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Figure 5; Time-frequency spread of common FIR filters. By changing the length of the filters in time, we eompute 
the time and frequency spreads for each type of the filters. For small values of frequency spread, Gaussian filters are good 
approximations of Mathieu functions (as shown also in [6]). 



when the frequency spread of sequences are very small, sampled Gaussians converge to the lower bound for 
maximally compact sequences. 

6. Conclusion 

Wc showed that contrary to continuous-time signals, for discrete sequences, there is a large gap between 
the Heiscnberg uncertainty bound and the time-frequency spread of maximally compact sequences. Note 
that our results arc consistent with the conclusions in [6] that no sequence can achieve the 1/4 Heisenberg 
uncertainty bound. The constructive way that wc propose in this research enables us to design sequences 
that are closest to the Heisenberg bound. Moreover, having Mathieu functions as the sprectra of maximally 
compact sequences, shows that again in contrary to continuous- time signals, sampled Gaussians are not the 
minimizers of the time-frequency spread, though they are close. 
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Appendix A. Proof of Lemmas 1 and 2 

Appendix A.l. Proof of Lemma 1 

Let X be maximally compact for a given time-spread A^(a;). 

^l{x) = A^dxl). 



AL(kl) 



< 



KM- 



(Al) 
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Since x is maximally compact, equality must be met in (A.l) to avoid contradiction, which amounts to 

XnX^+i = Ixnllxn+ile-''^, (/3 G [0, 27r) , Vn G Z. 

This condition is equivalent to (15). 

For maximally compact sequences, if A'^ strictly monotonically varies in function of A^^ , then fixing A'^ 
or A^^ is equivalent and proves the lemma. In the following lemma we show that for maximally compact 
sequences Af^ changes monotonically with A^^ . 

Lemma 4. For maximally compact sequences, A^ is a decreasing function of A^^. 

Proof. For proving this lemma, we use the dual formulation in (20). The feasible region of the dual problem 
is shown in Figure A. 6. We can write the dual as 

maximize c 

Ai,A2 

subject to A2 = c — a Ai , (A-2) 
A-\iB-X2lhO- 

Note that a changes between and 1 (see Figure A. 6). For a fixed a, the maximum c°p' is found by elevating 
the corresponding line A2 = c — aAi until it supports the feasible set (it is tangent to it). Since the feasible 
set is convex, as a grows (which means A^^ decreases), we need a higher elevation of the line to support 
the convex set, thus c°p* (equivalently A^J increases. □ 



Appendix A. 2. Proof of Lemma 2 

Consider the shift operator in (16). Since the shift operation does not change the norm of a sequence, 
we will assume a unit norm sequence without loss of generality. We can show that 

(j£(e-^-^(e^")),e-^-X(e^")) 

ZTT 

^ {j^X{en,Xien) i^{Xien,Xien) 

= /i„(a;) + i/, (A.3) 
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where we used the DTFT domain definition of the time center [6]: 



- ||^||2 

The proof for Lemma 2 — trivial for v € Z — is not obvious for arbitrary shifts. Let x be maximally 
compact with time center /i„(a;), then according to (A. 3), a;„_^ is centered at /U„(a;) + u and 



27rA^(x„_^) =27r 



auj aoj I 

= {-j,^X{en + X'ien, -JvX{en + X'{en) - 27r W{x) + 

+ (-jz.X(e^"),X'(e^")) + {X'(en, -]i^X{e^^)) - 2^ |m„(x) + z.|' 
= 2^ + (X'(e^"), X'(e^")) + 27r Real[/x„(2;)i/*] - 27r + 
= (X'(e^-),X'(e^-))-27rK(a;)|2 

= 2^A2(x). (A.4) 

This shows that time shift does not affect the time spread of a sequence. Thus, if a; is a maximally compact 
sequence, then Xn-n„(x) is also maximally compact (note that time shift does not change the frequency 
characteristics of the sequence). □ 

Appendix B. Proof of Theorem 2 

By using Lemmas 1 and 2, we can write problem (14) as 

^n,opt = minimize V n^a;^ 

subject to ^ XnXn+i = -i=L=, (B.l) 

nez 

We can rewrite (B.l) in a matrix form as a quadratically constrained quadratic program (QCQP) [17] 

minimize x^Ax 

X 

subject to x^Bx = a , (B.2) 
x^x = 1 , 



where A and B are defined in (19) and a = l/Vl + a^ . This problem can be further reformulated as 
follows: 

minimize tr{Axx'^) 

X 

subject to tr(Bxa;^) = a 
tv{xx'^) = 1 . 

Replacing xx^ by X, we can write equivalently 
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minimize tr{AX) 

subject to tr(BX) = a (B 3) 

tr(X) = 1 

X hO, rank(X) = 1 . 
We further relax the above formulation to reach the semi-definite program 

minimize tT(AX) 

X 

subject to tT{BX) = a (B-4) 
tr(X) = 1, XhO- 

In Lemma 5 we show that the semi-definite relaxation is tight. 

□ 

Lemma 5. The semi-definite relaxation (SDR) from (B.3) to (B.4) is tight. 

Proof. Shapiro and then Barnivok and Pataki [18, 19, 20, 21] show that if the SDP in (B.3) is feasible, then 



rank(X°P*) < [(\/8m + 1 - 1) /2J , (B.5) 

where m is the number of constraints of the SDP and X°p* is its optimal solution. For our semi-definite 
program in (B.3), m = 2. Thus, (B.5) implies that the solution has rank 1. Using this fact, one can see 
that the semi-definite relaxation is in fact tight. Note that from the nature of the problem, (B.3) is clearly 
feasible; we can always find a periodic signal in the Fourier domain with a unit norm and a desired frequency 
spread, although not having an optimal time spread. □ 



Appendix C. Proof of Lemma 3 

We use the following Lemma for the proof: 

Lemma 6. [22, 23] For a semi-definite program and its dual: If the primal is feasible and the dual is strictly 
feasible, then strong duality holds. 

It is easy to see the primal is feasible. One can always find a signal x with a certain frequency spread and 
norm one. Using this signal we can construct X = xx^ which shows the feasibility of the; primal. For the 
dual, one can use the Gershgorin's circle theorem and show that a sufiicient condition for A— Ai B — A2 / ^ 
to hold is A2 < — Ai and Ai > 0. Thus, the dual problem is strictly feasible. □ 



Appendix D. Proof of Theorem 3 

If a sequence is a solution to the dual SDP problem (20), the dual constraint is active. Therefore, 
maximally compact sequences lie on the boundary of the quadratic cone 

A - Ai B - A2 / t 0. 
A maximally compact sequence x is thus solution of the eigenvalue problem 

{A-\iB)x = A2£c, (D.l) 

where Ai and A2 are the dual variables of the SDP problem, and A2 is also the minimal eigenvalue oi A — \iB 
and X is the associated eigenvector (this can be also seen by forcing the derivative of the Lagrangian in (B.2) 
to zero). 
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This explicit link between the dual variables and the sequence, yields a differential equation for which 
the DTFT spectrum of maximally compact sequences is the solution. In the DTFT domain (D.l) becomes 
(expanding the matrix multiphcations) 

-X"{e^'^) - XiCos{uj)X{eJ'^) = A2X(e^"), 
^ X"(e^"') + (A2 + AiC0s(w))X(e^'^) = 0, (D.2) 

which is Mathieu's differential equation ([24]§20.1.1): 

+ (a-2q cos(2t)) . y{t) = 0. (D.3) 

The solutions of Mathieu's equation are called Mathieu functions, and they assume an odd and even 
form 

Mathieu's Cosine (even) ce{a,q;oj) , (D.4) 

Mathieu's Sine (odd) se(a, q; cj) . (D.5) 

Taking into account the periodicity of (D.2), it appears not all pairs of parameters {a,q) will lead to a 
periodic solution. Mathieu functions can be restricted to be 27r periodic: 

Definition 1. The solutions of Mathieu's harmonic differential equation — equation (D.3) with a solution y 
27r-periodic — are defined as 

Mathieu's harmonic Cosine (even, periodic) csmiq ', w) = ce{am{q),q ; w) , m e N. (D.6) 
Mathieu's harmonic Sine (odd, periodic) ssmiq ', w) = ce{bm{q),q oj) , m G N"*". (D.7) 

It is immediately visible that the spectrum of maximally compact sequences may only have the form 

^(e^"') 70 • ce™(-2Ai ; w/2) + 71 • se„(-2Ai ; w/2), for m G N+, 

X(e^") = 70 • ce™(-2Ai ; w/2), for m = , (D.8) 

for any constants 70 and 71 such that ||X(e''"|| = 1. More specifically, for any Ai > 0, the dual SDP problem 
can be posed and any solution would have the form (D.8) 

Charasteristic numbers of Mathieu's equation are ordered [25] , such that 

ao(-2Ai) < ai(-2Ai) < 6i(-2Ai) < 62(-2Ai) < a2(-2Ai) < • • • , Ai > , 
ao(-2Ai) < 6i(-2Ai) < oi(-2Ai) < 62(-2Ai) < a2(-2Ai) < • • • , Ai < , 

By (D.3) and with the substitution t u>/2 one obtains a„i(— 2Ai) = 4A2. Because A2 is the minimal 
eigenvalue, we conclude that m = 0. 

Note that this result validates the one in [26] which stated that asymptotically Mathieu functions mini- 
mize the time- frequency product. With a difcrent approach, we can establish that only Mathieu's harmonic 
cosine of order minimizes this product for any given frequency-spread. □ 

Appendix E. Proof of Theorem 4 

In order to prove the theorem we first provide the following lemma. 
Lemma 7. If 

A2<l--^1 + A2, (E.l) 

then P = A- XiB ~ X2I 0. 
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Proof. Note that if Ai =0, then the matrix P is positive-semi-definite with the given condition. So in the 

proof we will assume that Ai 7^ 0. 

Consider the following matrices Pi and P2 : 



Pi 



O-A2 -Ai 

-Ai/2 I-A2 -Ai/2 
-Ai/2 4-A2 -Ai/2 











l-Aa -Ai/2 
-Ai/2 4-A2 -Ai/2 





(E.2) 



Call I the set of eigenvalues of P, Ii the set of eigenvalues of P\ and I2 the set of eigenvalues of P2- It 

is trivial to sec that X = X\ U T2 . 

We show that if condition (E.l) is satisfied, then both P\ and P2 have positive eigenvalues. 

1. Pi 

One can decompose Pi as 



Pi = 







1 



-A2/2 -Ai/2 
-Ai/2 I-A2 -Ai/2 
-Ai/2 4-A2 



-Ai/2 











= D X P 



(E.3) 



Note that both D and P{'' are symmetric. Now define 

5,def^_l/2p^^l/2 

= L>-i/2 {dP^^ L>i/2 
= D'/^P^'^D'/^ 

First observe that S is symmetric, thus it has real eigenvalues. 

Lemma 8. Eigenvalues of S and Pi are equal. Further, if v is an eigenvector of S, then D^^^v is an 
eigenvector of Pi. 

Proof. If A is an eigenvalue of S corresponding to eigenvector v, then 

Sv = A^; 
^ D-y^PiDy^ = Xv 
^ PiDy^v = D^/^Xv 

=> Pi =A (r>V2^j 

□ 

According to Lemma 8, it sufSccs to consider eigenvalues of S; If S is positive-definite then all the 
eigenvalues of Pi are positive, and 

-A2 -X1/V2 

-X1/V2 I-A2 -Ai/2 
-Ai/2 4-A2 -Ai/2 
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Sylvester's criterion states that a symmetric matrix is positive definite if and only if its principal minors 
are all positive. We can use Gaussian elimination on the matrix S to compute its principal minors. 



"O-A2 -X1/V2 

Si -Ai/2 

S2 -Ai/2 

■•• 



(E.4) 



where si = 1 — A2 + Af /2A2 and S2 = 4 — A2 — Af /4si. In general one can write 



Note that 



Si+i = {i + iy-X2--^, i>i. 



Si+2-Si+\ = 2z + 3 + Aj- 



If Si+i > Si then > Sj+i- Thus if we have sq > and S2 > sij then by induction, Si > and thus 
S is positive-definite. We show that under condition (E.l), this is true. One can easily check that 
under this condition: 

(i) so > 0. 

.... Jl+Xf-l 

(n) si = ^ — 2^ — > 0. 

(iii) S2 — si = 3 > 0. 

This shows that under condition (E.l) S is positive definite and thus Pi has only positive eigenvalues. 
2. P2 : 

As P2 is already symmetric, we can immediately use Sylvester's criterion on it. One can write 



pU 



1- A2 


-Ai/2 








0" 





S2 


-Ai/2 














■S3 


-Ai/2 



















(E.5) 



where 



Sj+i = {i + if - A2 



4s,; 



i>l. 



It is easy to see that under condition (E.l), Sj > |Ai| /2 > 0: 
(i) 

= 1- A2 = 1- (1- ^TTa?) = TTTa? > ^ 

(ii) 

Si+i = (i + 1)2 _ A2 - ^ = + 2i + JTTAf - ^ 
Thus, by induction if Si > |Ai| /2, then 



Si+i >i^ + 2i + |Ai| 



4|Ai|/2 



i2 + 2i+|Ai|-^ 
|Ai| 



> 



> 0. 
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Thus, P-2 is also positive-definite. Putting these together, one can conclude that under condition (E.l), 
the matrix P is positive-definite. 

□ 

Note that condition (E.l) gives a sufficient (but not necessary) condition on the feasible set of the dual 
problem (20). Thus, it provides a lower bound for the maximum value of the dual. Consider the restricted 
dual problem 

maximize a Ai -|- A2 

^"^^ (E.6) 



subject to A2 < 1 — y 1 + Aj 



The solution to this problem is simply 1 — Vl — ct^. If we rewrite a in terms of , we have at the end 



> 1 ■ 



".opt - V 1 + a2 ■ 

This concludes the proof for the lower bound. □ 

Appendix F. Proof of Theorem 5 

It is easy to see that for small values of (equivalcntly a closer to 1), the maximum of the dual problem 
is achieved for large values of Ai (remember that Ai needs to be positive). We saw in Appendix D that 
for maximally compact sequences (i.e. sequences that result in the maximum of the dual problem) we have 
A2 = 1/4 ao(2Ai) (note that ao(— g) = ao{q)). McLachlan in [27] shows that for large enough values of q, we 
have 



„ 1 1 1 ^ 48 _i 848 ^ 4,752 126,752 ^ 

a^{q) = -2q + 2q-^+---q^ - ^q - ^q ^ " "^2^9^ - 

= -2q + 2qi + ^- ^q^ + 0{q-^) . 



(F.l) 



Thus, we have for large q, 

ao{q)<-2q + 2q^ + (F.2) 
After replacing q by 2Ai and ao(2A) by 4A2, we have 

A2 = Jao(2Ai)<-Ai + -i=V^-^. (F.3) 

Because this set contains the original feasible set of the dual problem, it will give an upper bound on the 
optimal value of the dual. In other words, < cr^, where 

cr^ = maximize a Ai + A2 

Ai ,A2 

1 r- 1 (^-^^ 
subject to A2 < — Ai H — ;=\/ Ai — ■ 

V2 16 

It is easy to see that the maximum of (F.4) is achieved for Ai = j^jz^- Replacing Ai in (F.4) and 
using the fact that A^ < ct^, leads to 



□ 

19 



References 



[1] W. Heisenberg, The cLctual content of quantum theoretical kinematics and mechanics, Physikalishche Z. 43 (1927) 172, 
(Translated) . 

[2] M. VetterU, J. Kovacevic, V. K. Goyal, Foundations of Signal Processing, 2012, http://www.fourierandwavelets.org/. 
[3] E. Breitenberger, Uncertainty measures and uncertainty relations for angle observables. Foundations of Physics 15 (3) 
(1985) 353-364. 

[4] K. V. Mardia, P. E. Jupp, Directional Statistics, Wiley, 2009. 

[5] J. Prestin, E. Quak, Optimal functions for a periodic uncertainty principle and multiresolution analysis, Proceedings of 

the Edinburgh Mathematical Society 42 (2) (1999) 225-242. 
[6] J. Prestin, E. Quak, H. Rauhut, K. Selig, On the connection of uncertainty principles for functions on the circle and on 

the real line, Journal of Fourier Analysis and Applications 9 (4) (2003) 387-409. 

[7] S. Massar, P. Spindel, Uncertainty relation for the discrete Fourier transform, Pliys. Rev. Lett. 100 (2008) 190401. 

[8] T. Przcbinda, V. DeBrunner, M. Ozaydin, Using a new uncertainty measure to determine optimal bases for signal repre- 
sentations, in: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1999. 

[9] D. L. Donolio, P. B. Stark, Uncertainty principles and signal recovery, SIAM Journal on Applied Mathematics 49 (3) 
(1989) 906-931. 

[10] A. Agaskar, Y. Lu, Uncertainty principles for signals defined on graphs: Bounds and characterizations, in: Acoustics, 
Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, 2012, pp. 3493 -3496. 

[11] Z. Khalid, S. Durrani, P. Sadeghi, R. Kennedy, Concentration uncertainty principles for signals on the unit sphere, in: 
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012, pp. 3717 —3720. 

[12] R. von Mises, Uber die "Ganzzahligkeit" der Atomgewichte uud vcrwaudtc Frageii, Physikalishche Z. 19 (1918) 490-500. 

[13] D. Slepian, Prolate spheroidal wave functions, Fourier analysis, and uncertainty. V:The discrete case 57 (5) (1978) 1371- 
1430. 

[14] A. Brown, P. Halmos, C. Pearcy, Commutators of operators on Hilbert space, Canad. J. Math 17 (1965) 695-708. 
[15] E. Schrodinger, About Heisenberg uncertainty relation. Proceedings of The Prussian Academy of Sciences XIX (1930) 
296-303. 

[16] M. Grant, S. Boyd, cvx: MATLAB software for disciplined convex programming, version 1.21 (2010). 
[17] S. Boyd, L. Vandenberghe, Convex optimization, Cambridge University Press, 2004. 

[18] A. Shapiro, Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis, Psychometrika 
47 (2) (1982) 187-199. 

[19] A. Barvinok, Problems of distance geometry and convex properties of quadratic maps. Discrete & Computational Geometry 
13 (1) (1995) 189-202. 

[20] G. Pataki, On the rank of extreme matrices in semidcfinite programs and the multiplicity of optimal eigenvalues, Mathe- 
matics of Operations Research (1998) 339-358. 

[21] Z. Luo, W. Ma, A. M. C. So, Y. Ye, S. Zhang, Semidcfinite relaxation of quadratic optimization problems, IEEE Signal 
Processing Magazine 27 (3) (2010) 20-34. 

[22] M. Todd, Semidcfinite optimization. Acta Numerica 10 (2001) 515-560. 

[23] M. Trnovska, Strong duality conditions in semidcfinite programming. Journal of electrical engineering 56 (12). 

[24] M. Abramowitz, I. Stegun, Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. 

National Bureau of Standards Applied Mathematics Series 55. Tenth Printing., ERIC, 1972. 
[25] A. Erdelyi, W. Magnus, F. Oberhettinger, F. Tricomi, H. Bateman, Higher transcendental functions. Vol. 3, McGraw-Hill 

New York, 1955. 

[26] S. Song Goh, C. A. Micchelli, Uncertainty principles in hilbert spaces. Journal of Fourier Analysis and Applications 8 
(2002) 335-374. 

[27] N. W. MacLachlan, Theory and Applications of Mathieu Functions, Dover Publications, 1964, p. 232. 



20 



